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ABSTRACT 

The nature and extent of variations in student 
attributions about performance in their courses were studied, and the 
relationships between the attributions and responses on certain items 
of a student ratings questionnaire were determined. Causal or 
predictive relationships among these variables were also 
investigated. Data were collected using: (1) the Teacher-Course 
Evaluawion Project questionnaire (TCEP) (described by K» Theall and 
others in 1987), with 47,732 respondents in 2,381 courses; (2) the 
Motivated Strategies for Learning Questionnaire of P. R. Pintrich et 
al. (1938); and (3) a self -evaluation questionnaire completed by 
faculty in from 352 to 1,200 classes. Attributions of performance 
were strongly related to expected grades as predicted by previous 
research and fluctuated with respect to other instructional variables 
in similar and predictable ways. Attributions of internally and 
externally oriented students were shown to be significantly 
different. Attributional (self-serving) bias was established, but 
found not to invalidate ratings. Attr. DUtions were shown to be 
effects and not causes. Faculty attributions also lollowed patterns 
suggested by previous literature. Ten tables presenting study data 
and six figures are included. Attachments provide the questionnaires 
and summaries of factor analyses of questionnaire data. A 52-item 
list of references is included. (SLD) 
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VARIABLES IN THE STUDY 

LOCUS OF CONTROL: 

INTERNAL VS. EXTERNAL 
GENERAL EVALUATION ITEMS: 

A = instructor 
B = course 
C = amount learned 
D = organization 
E = difficulty 
STUDENT ITEMS: 

F = content value 
G=pr:. preparation 
H = class level 
I =G.P.A. 
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RESEARCH QUESTIONS: 

1. Do attributional patterns vary as grades vary? 

2. Are attributional patterns for the variables of interest the same as the basic, 
attribution X expected grade pattern? 

3. Do attributions change across the levels of the variables of interest? 

. 4. Are the ratings of "internals" and "externals" significantly different on the 
variables of interest? 

5. Can attributional locus be predicted with the variableo oi interest? 

ADDITIONAL QUESTIONS: 

1. Are attributions "state" or "trait" variables? 

2. What is the pattern of faculty attributions about student performance? 

CONCLUSIONS: 

L Attributions are "effects", not "causes". 

2. Attributions become more external as grades decrease. 

3. Attributions are not affected by 'class level' or 'G.PA.'. 

4. Attributions become more external as ratings on other items "decrease". 

5. Attributional patterns change across the levels of the variables. 

6. Attributional results support the validity of student ratings. 

7. Attributional results do not support the notion cf "grade inflation". 

8. Attributional locus can be predicted with moderate success. 

9. Attributions are both 'state' and 'trait' variables. 

10. Teacher attributions change as teacher estimates of class quality change. 
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Attributions or retributions: student ratings ' 
and the perceived causes of performance. 

Michael Theall and Jennifer Franklin, Northeastern University 
Larry Ludlow, Boston College 



INTRODUCTION: 



When students expect to succeed, to perform adequately, or to 
fail in academic settings, what are their feelings about the 
causes of their performance? Do they hold themselves, others, 
or events and situations responsible? Do they always attribute 
their performance to the same factors, or do their attributions 
change, and if so, under what conditions? Finally, if these 
students are evaluating teachers and courses, can these 
attributions unreasonably color their responses on ratings 
questionnaires? 

The objectives of this study were: 1) to investigate the nature 
and extent of variations in student attributions about 
performance in their courses; 2) to assess the general 
relationships of attributions to responses on certain items of a 
student ratings questionnaire; and 3) to determine whether 
causal or predictive relationships exist among these variables. 

The attributions were classified as internal or external with 
further distinction being made between stable and unstable 
attributions, while the items taken from the evaluation 
instrument included overall ratings of course, instructor, and 
amount learned; expected grade and grade-point-average; course 
difficulty; class level; interest in course content; ex- 
pectations about the instructor; and course-related entry 
skills. 



THEORETICAL PERSPECTIVE: 



Attribution theory 

The largest compendium of attribution research and theory is a 
three volume set of books edited by Harvey, Ickes, & Kidd, 
(1975; 1978; 1981). Interestingly, though the volumes contain 
material important to the development of theoretical positions 
about academic attributions in the higher education context 
(e.g. Shaver, 1981), there are no chapters specifically ad- 
dressing the present issue. Lefcourt (1976) briefly discusses 
locus of control and academic performance (p. 69) but focuses on 
young learners concluding that "...the relationships 
between... locus of control and achievement behaviors are often 
riddled with inconsistent. . .results" (p. 71). 

General attribution theory (Weiner, 1974; 1979: see Table 1 for 
an array of the Weiner attributional dimensions) suggests that in 
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instances of poor performance or failure, the locus of causal 
attributions will shift toward externality. The theory has been 
tested in higher education situations with respect to 
attributions, affect, and expectations (Forsyth & McMillan. 
1981) and there has been some investigation of the relationship 
of causal attributions and expectancy of success (e.g., McMahan, 
1974, whose samples included primary, secondary, and college 
students). But the focus in these studies was on expectancy 
and on attributions either to test Weiner's model (the former 
study) or to relate attributions and expectancies 12 ability, 
effort, task, and luck (the latter study). Marsh, Cairns, 
Relich, Barnes, & Debus (1984) reviewed the literature of 
attributions and self-concept, focusing on the "self-serving 
bias" of attributions, and conducted a study with 248 fifth 
graders as subjects. They proposed a "controversial" conclusion 
which suggested that attributions were more complex than would 
be expected by the basic models, and stated, 

...dimensions of attribution that have been found in 
situational studies do not adequately represent dispo- 
sitional dimensions. .. .this conclusion. . .does not argue 
against the validity of. ..studies that support the Weiner 

model It does, however, challenge the assumption that 

...academic self -attributions can be adequately described 
by a single bi-polar dimension. . .or by the three bipolar 
dimensions proposed in the Weiner model. (p. 28) 

However, in these studies, student ratings of teacher perfor- 
mance were not specifically considered. 

Attributions and evaluations 




Crittenden and Norr (1973) viewed student evaluation as a 
"special case of person perception" (p. 143) cautioning against 
ranking faculty on global appraisals because such ratings 
reflect the interaction of student values and teacher 
behaviors. Owen & Froman (1977) examined student's perception 
of locus of control and a discrepancy score of GPA minus 
expected grade in course on the dependent measure of 13 student 
ratings items. Regression analysis provided a significant 
interaction whi '1 partially supported the hypothesis that locus 
reduces the validity of studenc ratings instruments. The weak- 
ness of thr study is the discrepancy score which, as measured, 
compares a specific expectation based on in-class and test per- 
formance against a standard (GPA) which is an 'average'. A 
better measure of discrepancy would be expected grade versus 
actual grade. Studies cited as evidence of the disconfirmed 
expectancy influence (e.g., Bausell & Magoon, 1972; Kennedy, 
1975) were different in a major respect: they dealt with dis- 
crepancy between expected and actual grades rather than expected 
grade and GPA. Though significant, the results of Owen & 
Froman 's regressions were not reported in terms of effect size. 
R or R-squar^d values were not provided for the three instances 
of incremen al validity which achieved significance, while two 
interaction /ere reported with R values of .15 and .24. Owen & 
Froman suggest the results "...represent small but systematic 
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sources of 'noise' in the rating of teachers" (p. 4). fhey then 
asked "If the results prove general izable, should we discount 
external students' ratings of their teachers?" (p. 4) 

Ames & Lau (1979), reported positive correlations between 
attributions an^J ratings (internal with high ratings and 
external with low), noting "impressive" attribution evidence for 
convergent and discriminant validity of ratings. Perry & 
Dickens (1984, 1987) investigated student-perceived control in 
the classroom while manipulating teacher expressiveness and 
nature of feedback. They reported that high non-contingent 
failure feedback reduced perceived control and that with 
expressive instruction, "...achievement deteriorated in 
accnrdance with the severity of students' uncontrol lability." 
(p. 291) Coleman & Keller (1978), found that the interaction 
of predicted locus and achievement was the best predictor of 
students' sel f -reported progress and "also cautioned against the 
use of self-reported progress ratings as a measure of course 
quality. Keller (1983) has also argued convincingly for the 
inclusion of motivational considerations such as expectancy for 
success and the development of internal student locus of control 
in the design of instruction.: Scott (1981) reported positive 
correlations between locus and overall effectiveness of 
instruction noting locus to be context-specific, and stated 
that the coincidence or non-coincidence of perceived and actual 
loci can influence global appraisals. These studies have es- 
tablished locus as an important factor in relation to achieve- 
ment as well as to understanding ratings. 

Course grades and ratings 

There is a large body of litenture on the general relationship 
of grades to ratings. This litvrature is relevant because 
attributions are often considered with respect to expected or 
actual grades as measures of performance. For example, Bausell 
& Magoon (1972) reported grade-based biases which they described 
as possibly "retributive" while Howard & Maxwell (1980, 1982) 
found the positive correlation between grades and ratings to be 
a statement supporting ratings validity. They said, 

...the relationship between grades and satisfaction 
might be viewed as an expected result of important 
causal relationships of other variables (student mo- 
tivation and progress in the course) with satisfaction 
and grades rather than simply evidence of contamination 
^ due to grading leniency. (p. 175) 

Expectancy violations and ratings 

Another group of related studies examined whether the 
expectations of students about instructors or grades affect 
ratings (Bates, 1987; Feldman, Saletsky, Sullivan & Theiss, 
1983; Gigliotti, 1987; Holmes, 1972; Kennedy, 1975; Painter 
& Granzin, 1972). The findings are somewhat mixed but general- 
ly indicate: 1) a relationship exists between expected grades 
and ratings; 2) that relationship is not sufficient to support 
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the theory that giving good grades guarantees high ratings 
(i.e., "grade-inflation"); 3) violation of expectations about 
instructors or grades can lower ratings with the particular 
finding that this effect is independent of the direction of the 
violation in grades; and 4) internal attributions are more often 
associated with positive outcomes (i.e., achievement) than are 
external attributions. 

Attributions, expected grades, and student ratings 

Theall (1986) reported preliminary analyses (crosstabulations) 
supporting the general theory that attributions become more 
external as expected grades decrease. This pattern of attribu- 
tions across grades remained constant across five levels 
of grade-point-average and across class levels from freshman 
through graduate. However, when overall ratings of instructors 
were added into the analysis, only the rating pattern for 
"average" instructors was similar to the expected grade-by- 
attribution pattern. As instructor ratings increased or 
decreased, the pattern changed, particularly with students 
reporting low expected grades. 

With the "best" instructors, the frequency of external 
attributions of 'D' and 'F' students was less than that in the 
sample (20.8% of all attributions were external). With the 
"worst" instructors, the frequency of 'D' and 'F' student 
external attributions was more than three times what was 
expected. In^^urnal attributions were less affected, the great- 
est differences occurring with 'A' students. Their internal 
attributions were more frequent than expected with "best" 
instructors and less frequent than expected with "worst" 
instructors. All analyses were crosstabluations and all 
contingency table chi-square values were significant at extreme 
alpha levels (in part, an artifact of sample size). The 
results sugc 5ted that students attributions are affected by 
factors other than simple success or failure and that these 
attributions are sophisticated because they take into ac- 
count, the quality of performance at both loci. Thus, in 
cases of failure, the inclination to 'blame' the instructor is 
tempered if the student perceives that the instructor did a good 
job, but reinforced if the perception is that the instructor did 
a poor job and is more culpable. 

Gigliotti & Buchtel (1989) carefully examined attributional bias 
and course evaluations, collecting expected grade information at 
the beginning of the term, evaluation data at the end of the 
term, and actual grades from the registrar after the end of the 
term. They classified expected and actual grades to produce 
four student groups (Hi/Hi, Lo/Lo/, Hi/Lo, Lo/Hi) and assessed 
attributional locus with tl evaluation instrument items (e.g., 
an item on how the course affected feelings about self) rather 
than specific items of a locus instrument. They also used demo- 
graphic variables (age, sex, father's education level, etc' for 
additional analysis. Their findings were as follows: 
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* grades can not be ruled out as a potential biasing factor 
with this effect apparent at the individual, rather than the 
class level; 

* expected grade and expectancy violation are ruled out as 
biasing factors; 

* instructor evaluations are minimally affected by the 'self- 
serving' bias reported in attribution research; 

* there is very little evidence to support the 'grading la- 
niency' hypothesis; 

* demographic and situational variables have a slight biasing 
effect and may also affect attributions; and 

* bias in student evaluations is small and acceptable if suf- 
ficient care is employed in the entire evaluation process. 

Gigliotti and Buchtel (1989) also note that- "From a practical 
standpoint, it appears 'that' there may be problems in'^inter- 
preting the course and instructor evz^luations." Given the 
problems many faculty and administrators have in interpreting 
and using even basic ratings information for teaching 
improvement or personnel decision making, (Franklin & Theall, 
1989) this caution seems well advised. 

Analysis of the locus of attributions thus provides results 
which support the validity and reliability of student ratings. 
From these analyses, one can conclude that students are valid as 
raters because they can assess instructional quality independent 
of even person variables, and that evaluation ratings are 
reliable because they will be consistent within classes (i.e., 
excellent/average/poor instruction will be rated as such whether 
students are internally or externally oriented) and across 
classes (i.e., results will be consistent across class levs'ls 
and cll ranges of grade point average). 

Limitations of previous studies 

In these studies, sample sizes have varied. Howard and Maxwell 
(1980) used the IDEA database (overall n = 200,000 +; unit of 
analysis was the course with n = 8,551 courses in the sample) 
anu Bausell and Hagoon (1972) used 12,000 individual responses 
from a local evaluation instrument. But these two studies 
focused on the relationship of grades to ratings. Marsh's 
(1984) sample contained 248 subjects but these were fifth 
graders rather than college students. Gigliotti & Buchtel's 
sample contained 691 college students. Theall 's (1986) analysis 
of data (11,384 individual responses from a locally developed 
instrument) was as part of the validation process for the 
ratings instrument rather than a separate study. Crittenden and 
Norr used a locally devised instrument in 52 classes with a 
total sample of 1,718 students. Other studies noted in the 
review had sample sizes from 76 to 759. Thus, most of the 
studies of attribution -related issues either used relatively 
small samples or were preliminary in nature. 
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METHODS: 



The present study expanded on the authors' previous analysis i>y 
increasing the number of variab"'es investigated, specifying 
questions to be addressed, adding more powerful methods of 
analysis (analysis of variance and discriminant analysis, and by 
using a much "'arger database {about 50,000 respondents). In the 
previous analysis, there were many instances where cell sizes 
for extreme conditions made interpretation difficult or 
impossible {e.g., there were no students who expected 'F' 
grades in courses rated "least difficult", and t.here were many 
instances of cells with fewer than 10 members). In the present 
database (which incorporates the original data) this problem 
occurs only once. The cell for those students who axpected 'F' 
grades in the "least difficult" courses coritains only 3 
members. All other cell sizes are 20 or more. 



Instrumentation 
Student data 



Data were collected with the Teacher-Course Evaluation Project 
(TCEP) questionnaire used in Theall 's (1986) analysis and 
described by Theall, Franklin, & Birdsdall (1987). The TCEP 
questionnaire (copy ir^Tuded as Attachment A) was designed for 
diagnostic/ teaching improvement evaluation with the additional 
capability to provide infor"!ation for student course selection, 
personnel decision making, and educational or institutional 
research. The questionnaire contains 22 specific items about 
instructor behaviors; 9 general or summary items; and eight 
items about the student. 



This study concentrated on: 

1) five of the general items 

* overall rating of the instructor (TCEP item # 31) 

* overall rating of the course (TCEP item iSf 32) 

* rating of the amount learned (TCEP item # 30j 

* rating of the course organization (TCEP item # 28) 

* rating of course difficulty (TCEP item # 24) and 

2) six of the student items 



* student entry level skills (TCEP item # 33) 

* perceived value of course content (TCEP itr-T. # 34) 

* students' class level (TCEP item # 38) 

* self-reported grade-point-average (TCEP item # 39) 

* self-reported expected grade (TCEP item # 40) 

* the students' performance attribr.cion (TCEP item 36) 
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The attribution item was phrased as follows: 

"The single most important factor determining the grade 
I expect to receive in this course has been: 

A. my ability 

B. my effort 

C. the instructor's teaching ability 

D. how difficult (or easy) the course was 

E. the other students in the class 

F. luck" 

The response options, classified according to Weiner's (1974; p. 
6) scheme (when possible), are presented in Table 1. 



TABLE 1 

Classification of attribution response items 



Item 



locus /stability /intentional ity 



A. 'my ability' internal stable 

B. 'my effort' internal unstable 

C. 'the instructor's...' external stable 

D. 'course difficulty' external unstable 

E. 'other students' external unstable 

F. 'luck' external unstable 



unintentional 
intentional 
unintentional 
unintentional 

unintentional 



The other items considered in the study were common to most 
student rating forms. Evaluative items had typical response 
scales (e.g., the overall course item used a 5-point scale 
ranging from "one of the best" through "about average" to "one 
of the worst"). Demographics items were categorical (e.g., class 
levels) or Likert-type (e.g., prior preparation and content 
value used 5-point "strongly agree" to "strongly disagree" 
scales) . 
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Data used to address the "trait" vs. "state" question (see the 
"Research questions and hypotheses" section which follows) were 
collected using the Motivated Strategies for Learning Question- 
naire (MSLQ; Pintrich, et. al., 1988). MSLQ items are phrased 
to refer to one course at a time (e.g., MSLQ #2, "If I want to 
get good grades in this course, it depends on what I do."). 
Students respond on a seven-point Likert scale with end poin+s 
"Not at all true of me" and "Very true of me" and a neutral 
central option. MSLQ items were rephrased to address overall 
opinions about academic performance (e.g., #2, "If I want to get 
good grades in my courses, it depends on what I do.") because 
the questionnaire was not administered at the end of a course 
and because its purpose was to provide information about "trait" 
attributions. The response scale was also changed. A six- 
point, forced choice scale was used with end points, "Very much 
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like me" to "Very much unlike me." The neutral point was 
omitted in order to force a choice between loci. With only 200 
respondents, it was felt that too much data would be lost to 
'neutral' statements. The items from section 2a (motivation 
scale > expectancy component > control beliefs > internal-ex- 
ternal reasons for success and failure) were used to determine 
intemality and externality. Copies of the original items, 
their paraphrased versions, and the paraphrased response scale 
are provided in Attachment B 

Faculty data 

A self -evaluation questionnaire for faculty (Attachment C) is 
available in the TCEP svstem and it contains identical in- 
structor behavior and ijmmary items but replaces the student 
information items with instructor demographics. The self- 
evaluation questionnaire also cont ns two items related to the 
study. A general rating of the c Jity of the class (item § 
40), and an attribution item (# 39). The class rating uses a 
5-point "among the best" to "among the worst" scale while the 
attribution item asks instructors to identify "The single most 
important factor determining student performance (as measured 
by grades) in this course..." 

Responses to this item and Weiner, (1974) categories are as 
follows: 



A. student ability 

B. student effort 

C. my teaching ability 

D. my effort 

E. course difficulty 

F. mainly luck 



[external /stabl e/uni ntenti onal ] 
[external/unstabl e/i ntenti onal ] 
[ i nternal /stabl e/uni ntenti onal ] 
[i nternal/unstabl e/i ntenti onal ] 
[external/stabl e/uni ntenti onal ] 
[external/unstable/unintentional] 



Sample sizes 



The student questionnaire has been used in 2,381 courses with 
about 48,500 respondents (47,732 was the usable sample). The 
self-evaluation was used in over 1200 classes (usable sample 
sizes ranged from 352 to 1200 depending on the item). In both 
cases, the samples included multiple sets of ratings from 
individuals (i.e. students who had rated several courses and 
instructors who had used the self-evaluation more than once). 
No attempt was made to identify, isolate, or remove these 
cases. 
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Data collection procedures 

All data except MSLQ attributions were collected as part of the 
normal process of teacher-course evaluation. Packets of 
materials were sent to each instructor and included: 1) quest- 
tionnaires and response sheets pre-printed with instructor and 
course information {both matching the course enrollment as 
provided by faculty, and/or department, and/or registrar); 
2) information and instructions for data collection; 3) the 
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self-evaluation form and return envelope; and 4) a form to be 
signed by the student monitor stating that data were collected 
according to instructions. Faculty gave packets to student 
monitors whom they selected. Monitors administered the eval- 
uation and returned materials to a specified campus drop-off. 

Data were coded to indicate irregularities (e.g., evaluations 
administered after a final exam rather than during the two-week, 
pre-finals period) and were omitted from analysis if the 
irregularity could have affected results. The TCEP data 
management software produced a datafile ready for statistical 
analysis using SPSSX (SPSS Inc., 1988). All analyses were done 
with SPSSX. 

The MSLQ was administered at two, public, campus locations: a 
cafeteria and a lounge area. These provided 187 of the 204 
surveys while a set of office areas provided the other 17 
surveys. Students were paid $1.00 each for their time and 
effort. The lead author and a graduate assistant collected 
the data by requesting indviduals or groups of students to 
participate and then, monitoring their progress until the 
surveys were completed and returned. Students worked alone and 
were paid when they returned completed surveys. The process 
took an average of 20 minutes per subject to cmplete. At any 
given time during the process there were from 1 to 15 surveys 
underwsv. 

Research questions and hypotheses 

In this study, the basic assumption is made that causal 
attributions are the result of students' pc-rceptions and 
interpretations of events in the instructional context. In 
other words, attributions are effects, not causes. The two 
basic variables are thus the attribution and the measure of the 
students' performance. Operationally, these are defined as 
students' responses to items 36 & 40 of the TCEP questionnaire. 
Each of the other variables in the analysis is also defined 
as students' responses to the corresponding TCEP item. 

There were five major questions to be answered in the study. 

1. What are the patterns of attributional ratings across the 
range of expected grades? 

Hypotheses., null = observed attributional frequencies will 

not be significantly different in any 
expected grade level than the sample 
frequencies (Int = 79.2%; Ext = 20.8%) 

rsrch. = externality will increase as grades 

decrease and observed frequencies will 
be significantly higher than expected 
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Z. What are the attribution patterns with respect to the 
following variables? 

(A) overall instructor item # 31 

(B) overall course item # 32 

(C) amount learned item # 30 

(D) course organization . item # 28 

(E) course difficulty item # 24 

(F) perceived value of 

course content item # 34 

(G) students' prior 
preparation item # 33 

(H) student's class item # 38 

(I) students academic performance 
overall [i.e., GPA] . item # 39? 

Hypotheses., null = there will be no significant differences 

between expected and observed attribution 
frequencies for variables (A) through (I) 

rsrch. = attributions will become more external 
as ratings become negative and observed 
external frequencies will be signifi- 
cantly higher than expected 

3. Is the attribution by expected grade pattern affected when 
each level of each variable above is added into the analysis? 

Hypotheses., null = observed attributional frequencies 

across the levels of variables (A) 
through (I) will not differ signi- 
cantly from the basic attribution 
by expected grade frequencies 

rsrch. = attributional frequencies will differ 
significantly, becoming more external 
as ratings on variables (A) through 
(I) become more negative 

4. Are there significant differences between internally and ex- 
ternally attributing students when they respond to the items 
(A) through (I) above? 

Hypotheses., null = there will be no significant differences 

between mean scores of internal and 
external groups on variables (A) 
through (I) above 

rsrch. = external means will be significantly 
lower than internal means on all items 

5. Can attributional locus be predicted by using one or more of 
the items above? 

Hypotheses., null « prediction of the locus of attributions 

will not be Improved over chance level 
through the use of variables (A) to (I) 
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rsrch. = use of expected grade and certain items 
in (A) th'^ough (I) above will signifi- 
cantly increase the percent of variance 
accounted for in a predictive equation 

Two further questions were addressed in an exploratory manner. 
They were: 1) "Are attributions about specific, individual courses 
different from attributions about overall academic performance? 
(i.e., can attributional locus be both a "state", and "trait" 
variable); and 2) "What are the relationships of faculty attri- 
butions about student performance to faculty ratings of their 
own performance, the quality of their courses, and the quality 
of the students in their courses? 

No formal hypotheses were generated with respect to these 
questions but the preliminary analysis (Theall, 1986) suggested 
some possible results. First, attributions about long-term 
academic performance would be slightly more internal than those 
made about specific courses, and that even if such was the case, 
specific course attributions would not be affected. In other 
words, attributions vary across different expected grades in the 
same ways regardless of the long-term academic performance of 
the individual or the individual's attributions about that 
performance. Thus, individuals may have different loci of 
attribution in the two situations. 

The second question was not addressed in earlier analyses 
but it was expected that faculty attributions would be largely 
external, that is, that they would place the responsibility for 
student performance on students. New instructors or TAs were 
expected to display more internality. The relationship between 
attributions and other variables were not estimated. 

Data analysis and limitations 

All analyses were done using SPSSX (SPSS Inc., 1988). Indivi- 
dual responses to the various TCEP items were analyzed. But 
it is generally agreed that at least in validation studies of 
evaluation instruments, the 'class' should be used as the unit 
of analysis (e.g., Abrami, 1981; Marsh, 1987). Class means are 
required for factor analyses or similar single procedures as 
well as for combinatory procedures such as meta-analysis (i.e., 
as Whitely & Doyle [1976, p. 243] would use to investigate 
"between-class ...trait occurrence-factors"). 

When TCEP underwent validation, analyses using both class and 
individual responses were used. Validation reports were produced 
for internal purposes but not published separately, although some 
of the results were reported by Theall (1986). Results of 
factor, regression, and other analyses were identical. In this 
study, however, it would have been meaningless to calculate a 
'mean attribution score' for each class. This data is more the 
type that Whitely & Doyle (1976, p. 243) say can be analyzed 
for "within class ...trait usage factors." Further, there was 
but one attribution item and it used a nominal level response 
O scale. The nature of attributions as personal decisions in this 
ERIC 
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data makes analysis of individual responses intuitively 
appealing as well. 

Additional support for the use of individual data comes from 
Gigliotti and Buchtel (1989) who note that the "...potential 
biases we observe at the individual level may not generalize to 
the class level... " 

Finally, attributional data can be grouped for analysis and used 
as control variables in analyses of other data which are more 
amenable to interpretation as mean scores. An example wo'jld be 
the use of internal/external groupings on a class-by-class basis 
to categorize class means on 'overall instructor' and 'amount 
learned' items so that these could be analyzed with respect to 
specific behavior items. The objectives of such studies would 
be to determine whether certain teaching strategies benefit one 
group over the other and/or result in different overall ' " 
ratings. The problem here, is that instances of classes being 
predominantly 'external' are extremely rare. In the present 
database, the internal/external frequencies were 79.2% and 20.8% 
(respectively). Any instances of predominantly 'external' 
classes would also have included unusual pecentages of 'F' 
grades AND an equally unusual number of 'worst' ratings for the 
instructor. Cell sizes from the contingency tables showed that 
such cases did not exist in this study's data 

Another problem with the data was the difference in the sizes of 
the groups (approximately 37,000 internal and 10,000 external) 
with respect to the group variances. In the discriminant 
analyses. Box's M results were all significant at <.000 levels, 
while in the analyses of variance, Bartlett-Box figures were 
also significant at extreme levels. Whether this violation of 
the basic 'larger group - larger variance' premise seriously 
restricts the useful Iness of the results is hard to determine 
but, given the size of the sample, the assumption is made that 
the effect was minimal. 

The last data issue of note is limited range. Consider the 
variables "expected grade" and "overall instructor rating", 
for bxample. A crosstabulation analysis of these variables 
produces a 'chi square' value of 2682.2030 (sig. at <. 00001). 
As this result suggests, the observed frequencies in the whole 
sample differ from the expected by a. substantial margin. Yet a 
correlational analysis of the samn variables, though it produces 
a significant 'Pearson r' (.2189; alpha « <.000), accounts for 
only 4.79% of the variance. A review of the data shows why 
this is so. It is because over 90% of the data are found near 
the positive ends of their respective scales. Over ninety-five 
(95.9) percent of the expected grades are 'A', 'B', or 'C while 
over eighty-eight (88.1) percent of the instructor ratings are 
in the top three categories. These data differ only marginally 
in their observed and expected frequencies thus the range of 
values is considerably restricted. Positive skews in both 
grades and overall instructor ratings are to be expected but 
their co-occurrence makes analysis and interpretation more 
difficult. More will be said about the meaning of these analyses 
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in the discussion section below. 



RESULTS: 



Power of statistical tests 

There is a great deal of data pi-esented in this section. In 
order to simplify the presentation, a categorical statement 
can be made about the power of the statistical tests employed 
in the analyses. It is this: based on Cohen's (1977) tables, 
the power of all tests employed was .90 or higher. This, of 
course, is a function of the size of the sample and carries with 
it, the caution that even small effects will be significant. 
Thus, despite the fact that literally every test produced ■ 
statistically significant results (and these, at probabilities 
of .000 or less) the educational/social/psychological 
significance of the findings niust be realistically and carefully 
assessed. 

The research questions 

Question #1. ... The null hypothesis is rejected in favor of 

the alternative hypothesis. 
Observed attributional frequencies differed significantly from 
expected frequencies, (chi square = 4222.67; df = 25; alpha = 
<. 00000; eta = .263). In order to display these (and in fact all 
crosstabulation results) more clearly, a graphic system has 
been produced. Figure 1 uses that system and an explanation is 
provided below. 



insert Figure 1 about here 



The vertical axis contains the expected grades 'A' through 'F'. 
The horizontal axis contains a scale based on the ratio of 
the percentages of observed and expected frequencies for the 
attributions. For example, 14% of the students expecting 'A' 
grades made external attributions. The percentage of external 
attributions in the overall sample was 21%. A ratio of observed 
to expected would be 14/21 or its decimal equivalent, .67. 

On the horizontal axis, the values range from "0" (at left, 
indicating no observed occurrences) to "1.0" (the vertical line 
near mid-graph indicating equal expected and observed frequen- 
cies) to "5.0" (far right, indicating 5 times as many observed 
occurrences as expected). 

For example, if the observed and expected percentages of 
external attributions at the grade of 'A' were the same, the 
resulting ratio in this analysis would be 21/21 (or its 
equivalent, 1.0). Thus the data point for that ratio would be 
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on the "1.0" vertical line. If expected and observed 
percentages were the same at each grade level, a vertical line 
at "1.0" would result. Therefore the plotted lines deviate from 
the vertical line at "1.0" to the extent that observed and 
expected ratios are different. Also, the numbers along the 
horizont:»l line are at ratio level (e.g., '2.5' means 
two-and-one-half times as many attributions as expected). 

The sample internal and external values were 79% and 21% res- 
pectively. In Figure 1, external attributions were fewer than 
expected whan the expected grade was 'A'; almost similar at 
'B'; about 1.7 times mor;» frequent than expected at 'C; 2.5 
times more frequent than expected at 'D'; -md 2.65 times more 
frequent than expected at 'F'. Internal attributions were 
negligibly greater than expected at 'A'; exactly as expected at 
'B'; less than expected (.85) at 'C; and even less (.67 and 
.68) at 'D' and 'F' (respectively). 

Question #2. ... The null hypotheses are rejected in ravor 
of the alternative hypotheses for variables 
'A', 'B', 'C, 'D', 'F', and 'G'; 
rejected with r&;>ervations for variaDle 'E'; 
and rejected with strong reservations for 
variables 'H' and 'I'. 

Table 2 presents statistical results for the variables in 

alphabetical order. 



TABLE 2 



Crosstabulations statistics for variables 'A' through 'I' 



variable 


2 

'Chi ' 


d.f. 


sig. 


'eta' 


'A'.. 


overall instructor 


3581.60 


20 


all 


.227 


'B'.. 


overall course 


4660.08 


20 


values 


.267 


'C. 


amount learned 


4746.20 


20 


<. 00001 


.261 


'D'.. 


course organization 


3235.83 


20 




.227 


'E'... 


course difficulty 


1816.28 


20 




.155 


'F'.. 


course content value 


2559.95 


20 




.221 


'G'.. 


prior preparation 


3251.54 


20 




.215 


'H'.. 


class level 


784.31 


25 




.059 


'I'.. 


G. P. A. 


551.88 


20 




.079 



The graphic displays of re.'uslts for variables "A', 'B', 'C, 'D', 
'F', and 'G' are so similar to that shown in Figure 1 that 
figures for these variables are unnecessary. In all cases, 
external attributions are less frequent than expected at the 
positive end of the variable's response scale and much more 
frequent than expected at the negative end. Internal 
attributions are slightly more frequent than expected at the 
positive end of the scale and somewhat less frequent than 
expected at the negative end. 
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Figures 2 and 3 present results for variables 'E' and 'H'. 
Figure 2 (variable 'E'.. course difficulty) shows that both 
internal and external attributions were very close to expected 
levels when courses were rated "extremely easy", "easier than 
average", or "about average" but differed from expected values 
when courses were rated "more difficult than average" or 
"extremely difficult". (Note that the vertical axis contains 
the levels of the variable in question. Later, in discussing 
question 3, graphic displays will all use 'expected grade' 
values on the vertical axis.) The percentages of ratings 



insert Figure 2 about here 



on the difficulty item were unevenly distributed with 51% being 
"average", 30% being "more difficult", and 12% being "easier", 
but with only 5% being "most difficult" and 3% being "easiest". 

Figure 3 presents results for variable 'V (G. P. A.). These 
results are very similar to those for variable 'H' (class 
level). In both cases, the expected and observed percentages 
of internal and external attributions are very similar and thus, 
the plotted graphs do not vary far from the "1.0" line. This 
graph demonstrates the differences between these, and the other 
variables as shown in the 'chi square' and 'eta' columns of 
Table 2. In effect, the attributions of freshman are not really 
much different than those of graduate students and the attri- 
butions of students with G.P.A.s of 3.26 to 4.0 were really 
not much different from those of students with G.P.A.s of 1.0 to 
1.76. 



insert Figure 3 about here 



Question #3. ... The null hypotheses are rejected in i\.'or of 
the alternative hypotheses for variables 'A' 
through 'D', and variables 'F' and 'G'; 
rejected with reservations f^^r variable 'E'; 
and rejected with strong reservations for 
variables 'H' and 'I'. 

The statistics and graphs for variable 'A' (overall instructor 
rating) will be used as models for all the variables accepted 
without reservation. In Table 3, statistics are presented for 
five analyses: one for each level of the instructor rating item. 
Each analysis was of the expected grade-by-attribution pattern 
for one level of the instructor rating. In addition to the 
statistics, the distribution of responses across the five levels 
of the item are also provided. 



22 



ATTRIBUTIONS BY COURSE DIFFICULTY 



DIFFICULTY 

EASIEST 



AVERAGE 



HARDEST 



0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 
OBSERVED VS. EXPECTED ATTRIBUTIONS RATIO 



Internals 



Hi 



Externals 




Expected 



Based on sample n»47,732 
Internal«79%, External»21% 



Figure 2 



ATTRIBUTIONS BY GP.A 



G.P.A. 

3.26-4.00 
2.76-3.26 



2.26-2.76 -f- 



1.76-2.26 — 



1.00-1.76 



7f 



f 

I 
I 

i 



0.6 1 1.6 2 2.6 3 3.6 4 4.6 
OBSERVED VS. EXPECTED ATTRIBUTIONS RATIO 




Internals Externals 



Based on sample n=47,732 
lnternal=79%, Externah21% 



Figure 3 



ERIC 



25 



26 



Attributions... Theall, Franklin, SLu'^low... p. 16 



TABLE 3 



Crosstabulations of attribution by expected grade 
for five levels of instructor ratings 
('eta'= % of var. acctd. for; % = percent of whole sample) 



level 


2 

'chi ' 


d.f. 


sig. 


'eta' 


% 


"one of the best" 


759.38 


25 


all 


.174 


31.3 


"better than average" 


979.14- 


25 


val ues 


.212 


33.7 


"about average" 


1048.98 


25 


<. 00001 


.261 


22.8 


"worse than average" 


461.58 . 


25 




.266 


8.1 


"one of the worst" 


339.00 


25 




.284 


4.2 



Figure 4 presents graphs for these five analyses side-by-side. 
The graphic displays dramatically demonstrate how attributions 
change when the five different levels of instructor performance 
are taken into account. 



insert Figure 4 about here 



Of particular interest in Figure 4 is the movement of the 
graphs. External attributions are 1.2 times more frequent than 
expected for 'A' students when the instructor is "worst" but 
only .65 the expected value when the instructor is "best". 
With 'D' and 'F' students, attributions are 4 t'mes as frequent 
as expected with "worst" instructors but only 1.3 times as 
frequent with "best" instructors. Thus, even when failing, 
externally oriented students recognize, and differentiate 
between good and bad instruction. Internally oriented students 
make similar distinctions. With "best" instructors, their 
attributions are very close to expected values, but with "worst" 
instructors, all observed internal attributions are less 
frequent than expected. 

Also worthy of note is the fact that for "average" instructors, 
the graph takes the shape of the normal, attribution-by-expecteu 
grade graph, and that the intermediate ("better" and "worse") 
ratings produce patterns which fall between "average" and the 
appropriate semantic ends of the scale. In other words, the 
"better than average" ratings graph falls between "average" and 
"best" while the "worse than average" falls between "average" 
and "worst". 

Variables 'B', 'C, 'D', 'F', and 'G' all present similar 
statistics and similar patterns of attributions. Their 
"average" ratings produce graphs resembling the basic graph of 
attributions -by-expected grades; positive ratings are associated 
with movement away from externality; and negative ratings are 
associated with sharp increases in external attributions. 
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Table 4 presents statistics for the 'difficulty' item. 



TABLE 4 



Crosstabulations of attribution by expected grade for 
five levels of cov rse difficulty 



1 evGl 


2 

'chi ' 


d.f. 


sig. 


'eta' 


% 

dist. 


"extremely easy" 


119.85 


25 


all 


.181 


2.9 


"easier than average" 


322.16 


25 


values 


.175 


11.6 


"about average" 


1445.32 


25 


<. 00001 


.216 


50.9 


"more diff. than avg." 


1280.67 


25 




.269 


29.5 


"extremely difficult" 


321.81 


25 




.318 


• 5.0 



"Course difficulty (variable 'E)' presents an intermediate 
result similar to that reported in Question #1. It also 
presents an interpretation problem because, unlike the other 
ratings items, it is not value-loaded. Courses which are easy 
or difficult can also be among the best or worst. Factor 
analyses of evaluation instruments have consistently reported 
workload/difficulty as a separate factor (see Marsh, 1987, p. 266 
for an array of evaluation factors reported by various 
researchers.) The TCEP instrument has been factor analyzed 
either as a whole or in part (e.g. analyses by class level, GPA, 
etc.) over thirty times (Theall, 1986) and every analysis 
(including that of the present data done with both individual 
and class as the units of analysis) has produced workload/ 
difficulty as a separate factor. (A set of TCEP factor analysis 
results and factor reliabilities is included with the TCEP 
instrument in Attachment 'A'.) In these analyses the 
correlations between the workload/difficulty items and the 
overall ratings of teacher, course, and amount learned ranged 
between -.14 and +.18. In fact, the only TCEP items which 
correlated to workload or difficulty in any meaningful way 
(i.e., correlations of .25 to .30) were those concerned with 
course organization, the instructor's ability to present 
information, and test difficulty. These relationships are both 
logical and. predictable. 

Figure 5 displays the graphs for all levels of difficulty. The 
five graphs are quite similar to the basic attribution-by-ex- 
pected grade pattern and to each other, but raise certain 
questions. Why, for example, are the "easiest" and "most dif- 
ficult" graphs so similar, and why, with "average" and "easier 
than average" courses, do attributions cut back toward the "1.0" 
line rather than continuing outward as in the patterns of other 
variables? The discussion section will address these issues in 
more depth. 
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insert Figure 5 about here 



It has been noted that the graph of the attribution-by-class 
level was very similar to the graph of attribution-by-grade- 
point average (see Figure 3 and text on p. 12). Individual 
graphs of the levels of each of these variables are not 
presented because they are s-> similar to each other and to the 
basic attribution-by-expected grade graph. Tables 5 and 6 
present the statistics for these two variables. 



TABLE 5 

Crosstabulations of attribution by expected grade 
for six student classes 



class level 



'chi ' 


d.f. 


sig. 


'eta' 


% 


1692.01 


25 


all 


.311 


26.9 


585.28 


25 


values 


.265 


13.7 


742.00 


25 


<. 00001 


.277 


15.3 


754.55 


25 




.284 


14.3 


876.49 


25 




.281 


15.7 


531.57 


25 




.246 


14.1 



"freshmen" 
"sophomores" 
"middl ers" (see *) 
"juniors" 
"seniors" 
"graduates" 

* = cooperative education plar takes five academic years 



TABLE 6 



Crosstabulations of attribution by expected grade 
for five levels of grade-point average 



G.P.A. 




2 

'chi ' 


d.f. 


sig. 


'eta' 


% 


1.00 - 


1.75 


278.00 


25 


all 


.397 


2.4 


1.76 - 


2.25 


418.41 


25 


values 


.255 


11.0 


2.26 - 


2.75 


1137.28 


25 


<. 00001 


.281 


25.0 


2.76 - 


3.25 


1467.35 


25 




.261 


35.8 


3.26 - 


4.00 


1013.32 


25 




.257 


25.8 



Note the 'chi squares' and effect sizes (eta) for freshmen and 
the 1.0-1.76 GPA levels. The freshmen constitute the largest 
percentage of the sample (26.9) and have the largest 'chi' and 
'eta' values. Students reporting the lowest GPAs are the 
smallest group (2.4%), and have the highest 'eta' value but the 
smallest 'chi' value. Crosstabulation of GPA by class reveals 
an interesting (and possibly frightening) number. 56.5% of the 
students reporting GPAs of 1.0 to 1.76 were freshmen! The im- 
^ plications about expectancy for success and for retention 
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efforts are clear. Interestingly, freshmen and/or those with 
low GPAs are not over-represented among externally attributing 
students. Apparently other factors affect the locus of 
attributions more than these two variables. 

Question # 4. ... The null hypotheses are rejected in favor of 
the alternatives for 'expected grade'; 
for variables 'A' through 'G'; 
and for variable 'I', 
(variable 'H'.. 'class'., was not used because its response 
scale was nominal rather than interval: i.e., there is no 'mean 
score' for the variable 'class'... although some faculty have 
reported that they have taught "mean classes") 

Analyses of variance were conducted to investigate differences 
between external.*;' and internals' mean scores on each of the 
variables in question. Table 7 presents all the results.- 



TABLE 7 

Analyses of variance for all variables 
groups = internal & external attributionr 

(note: d.f. in all cases = 1) 



variable 



SS / MS F 



sig. 



r/m 



expected grade 

overall instructor 
overall course 
amount learned 
organization 
difficulty 
content value 
prior preparation 
GPA 



'A'.. 
'B'.. 
'C. 
'D'.. 
'E'.. 
'F'.. 
'G'.. 
'I'.. 



2515.8 


1787.7 


all 


>.80 


1330.5 


1189.1 


values 


>.80 


1153.1 


1170.6 


<. 00004 


>.80 


1074.9 


1178.7 




>.80 


1070.9 


1032.3 




>.S0 


?.40.2 


348.4 




>.80 


1217.4 


1066.3 




>.80 


1465 8 


1098.5 




>.80 


642.2 


440 4 




>.80 



* = measure of effect size taken from Friedman (1972, .p 279); 
.80 is the highest value provided in effect size table 



These results are particularly powerful. The means of external- 
ly attributing students were significantly lower than those of 
internally attributing students on all the variables above. A 
note of caution is necessary here, because the results can be 
incorrectly interpreted. It is NOT the case that external 
students give lower ratings. It is not even correct to say 
that attributions are HADE external by these variables. We can 
only say that the two groups ara significantly different. 

Question #5. ... the null hypothesis is rejected in favor of 
the alternative hypothesis with some 
^ reservations. 
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In order to investigate the possible causal relationships 
between the above variables and the locuj. of attribution, a 
series of discriminant analysis was conducted. These attempted 
to predict internal /external group membership using two or more 
of the above variables. Discriminant analyses attempted to 
predict individual membership in: 1) internal or external 
groups; and 2) internal/stable, internal/unstable, external/ 
stable, or external/unstable groups (see the chart on p. 5). 
Predictor variables were expected grade and 'A' through 'G' 
above used as selected pairs (e.g., expected grade and amount 
learned) or all included. Some analyses were stepwise, some 
used a command which forced the sequence of entry into the 
predictive equation. 

Table 8 has three parts. Part I presents results from an analy- 
sis using expected grade and variables 'A' through 'H' in the 
following order: expected grade; amount learned; prior prepa- 
ration; difficulty; value of content; organization; overall 
instructor; and. overall course. The analysis attempted to 
predict membership in one of four attribution category groups: 

1) internal -stable .. attributions to ability of self; 

2) internal -unstable .. attributions to effort of self; 

3) external -stable .. attributions to instructor's ability; 

4) external -unstable .. attributions to difficulty, luck, or 

other students in the class. 



TABLE 8 

RESULTS OF DISCRIMINANT ANALYSIS 



Part I .. Functions and summary statistics 

2 

Function Wilks' Lambda 'chi ' d.f. sig. cum. % var. 



1 


.8622 


7229.9 


24 


<. 00004 


11.69 


2 


.9770 


1135.3 


14 


<. 00004 


13.53 


3 


.9954 


224.4 


5 


<. 00004 


13.78 



The discriminant analysis produced three functions, each sig- 
nificant beyond .00004, and together, accounting for 13.78% of 
the variance (calculated as the sum of the squares of the 
cannonical correlations), an effect. of relatively robust size, 
particularly considering the variance inherent in a sample of 
this size. 

In order to determine which variables were the most powerful 
predictors, the group centroids in the three functions were 
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reviewed along with the correlations of the individual items 
within the functions. Table 8 (Part II) presents these 
figures. 



TABLE 8 

RESULTS OF DISCRIMINANT ANALYSIS 



Part II .. Correlations within functions; group centroids 

Correlations 

Variable Function 1 Function 2 Function 3 



overall course 


.69921 


-.55256 


-.06147 


amount learned 


.67365 


-.49682 


-.29792 


overall instructor 


•61023 


-.31098 


-.38480 


content value 


.58630 


-.15229 


-.15955 


organization 


. *?S334 


-.19990 


-.40185 


prior preparation 


.54(;,42 


-.29035 


.44326 


expected grade 


.56'j41 


.64381 


-.22439 


difficulty 


.35405 


.09354 


.60143 



Part II .. Discriminant functions at group means (centroids) 
Group Function 1 Function 2 Function 3 



1 external/unstable -.91710 .20025 .00948 

2 external/stable -.30963 -.31331 .09474 

3 internal/unstable .08587 -.02671 -.06056 

4 internal/stable .34218 .11220 .08085 



Function 1 is clearly defined with all the predictor variables 
correlating to the function with some strength and in the same 
direction. The distance between centroids particularly separates 
groups 1 and 4. Given the relationships between positive 
ratings and internality, the combined tables indicate that as 
predictor variable ratings increase, more internal attributions 
are made. The strong negative centroid for the external/ 
unstable group (almost 1 standard deviation below 0.0, the 
center of the scale) further defines the internal/external by 
high/low ratings relationship. 

Function 2 is much less powerful and clear. The negative 
correlations of most of the predictor variables and the strong 
positive correlation of expected grade may be an artifact of the 
first function. That is, in order for the functions to be 
discrete, items can not correlate strongly with both. This can 
be taken as evidenca of the discriminant validity of the ratings 
items. However, this does not explain the correlations for 
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expected grade, the only variable which is not evaluative. The 
two groups most separated by this function are external/unstable 
and external/stable. Perhaps what little power is left after 
Function 1, is discriminating between stable and unstable 
attributions. In other words, within the external group, 
membership in the stable or unstable categories is best 
predicted by expected grade and the direction of this prediction 
is opposite that which results from a prediction based on the 
other variables. 

Function 3, though significant, displays too little power or 
centroid separation to warrant clear interpretation. The most 
interesting fact about this function is that the only two 
variables which correlate positively, also correlate strongly 
and are logically related (i.e., ratings of course difficulty 
and students' prior preparation). 

Table 8 (Part III) shows the "hit ratio", that is, the accuracy 
of predicted versus actual group membership and presents the 
overall percent of correctly classified cases. 



TABLE 8 

RESULTS OF DISCRIMINANT ANA'.YSIS 



Part III .. Predicted and actual group membership 

predicted membership % 



Actual group 


cases 


gp 1 


gp 2 


gp 3 


gp 4 


ext. /unstable 


5153 


* 50.0% 


9.4% 


26.2% 


14.4% 


ext. /stable 


5497 


23.5% 


* 14.8% 


33.7% 


28.0% 


int. /unstable 


28528 


15.8% 


6.3% 


* 42.0% 


35.9% 


int./stable 


12303 


13.6% 


5.3% 


29.3% * 


51.7% 



Percent of "grouped" cases correctly classified = 42.24% 



The interpretation of predictive power of this analysis is most 
simply stated as a question of now much better than chance one's 
prediction is. There are four groups in the dependent variable. 
All else being equal, the chance of correct prediction would be 
25%. But with unequal group sizes, this changes to the per- 
centage of th" sample the group contains. In this data. Group 1 
contains 10.7% ; Group 2, 11.5%; Group 3, 54.1%; and Group 4, 
26%. To determine the overall success of the prediction, a 
"proportional chance criterion" (Hair, Anderson, & Tatham, 1987, 
p. 9) was calculated by summing the squares of the group 
percentages (this equals 38.5%) and adding 25% of that figure to 
itself. The resulting figure is 48.12%. A "hit ratio" of 
48.12% can thus be classified as "good". The "hit ratio" in 
^ this prediction was 42.24%: better than chance, but not 
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high enough to achieve the "good" criterion. Clearly, the "hit 
ratio" for group 2 was the major factor in failure to reach the 
criterion. Group 2 represents attributions to the external and 
stable attribution choice "the instructor's ability". This 
issue will be noted again in the discussion section. 

Additional analyses 

Two exploratory questions were noted earlier. They addressed 
whether attributions about long-term performance were the same 
as those about specific course performance, and what were the 
patterns of faculty attributions about student performance. 

Long-term attributions 

To assess long-term attributions, the Motivated Strategies for 
Learning Questionnaire (MSLQ; Pintrich et. a]., 1988) was admin- 
istered to 200 student volunteers. Eight items from the MSLQ 
were used to assess attributional locus. All items camG from 
Section 2b (internal -external reasons for success-failure). 
Four of the items were worded so that the attribution was 
internal (e.g., #2.. "If I want to get good grades in my 
courses, it depends on what I do."). Four item were worded with 
external attributions (e.g., #28 "If I get poor grades in my 
courses, it's because the instructors don't like students like 
me.") Additionally, the internal and external items each had 
two success statements and iwo failure statements. (See 
Attachment B for the items) The concern here, was only with 
locus in the grouped eight items and in each subgrouping. 

The frequencies of responses are shown in Table 9. 



TABLE 9 

Frequencies of internal and external responses in the MSLQ 



item type(s) 



all items (total) 

internally worded items 
externally worded items 

success -worded items 
fail are-worded items 

internal /success -worded items 
internal/failure-worded items 

external/success-worded items 
external/failure-worded items 



number % internal % external 



8 


67.8% 


32.2% 


4 


91.5% 


8.5% 


4 


44.2% 


55.8% 


4 


66.4% 


33.6% 


4 


69.4% 


30.6% 


2 


96.3% 


3.7% 


2 


86.8% 


13.2% 


2 


36.5% 


63.5% 


2 


51.9% 


48.1% 
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The differences associated with item wordings make interpreta- 
tion of these results somewhat difficult. The results suggest 
that item wording is important: the internality or externality 
of the wording exerting the greatest influence. Success and 
failure wordings have less influence. But the response scale 
for all items (a forced choice, 6-point scale) would have 
allowed internally oriented students to respond to the exter- 
nally worded items with the "not at all like me" option. 
Students' reasons for not choosing ^his option can not be de- 
termined at this time. 

Faculty attributions and attitudes 

Faculty attributions about student performance and their eval- 
uations of themselves and their students were assessed by 
analyzing items from the TCEP self-evaluation questionnaire. 
Crosstabulations of attributions w-rs made against three other 
variables: 1) self-evaluation of instructor performance; 
2) instructor evaluation of course quality; and 3) instructor 
evaluation of the overall quality of the students in the class. 
One other analysis was done to determine whether demographics 
might have affected the attributions. This analysis 
crosstabulated attribution and instructor rank. Table 10 
presents statistics for these four crosstabulations. 



TABLE 10 

Crosstabulations of faculty attributions about student 
performance with facul+y ratings of self, course, and the 
quality of students in their class 

2 

item 'chi ' d.f. sig. 'eta' 



self rating 

course rating 

rating of student quality 

instructor rank 



18.74 


20 


.22587 


.108 


33.71 


20 


.02816 


.136 


62.49 


20 


.OOOOi 


.253 


70.79 


25 


.00001 


.107 



Three of the four crosstabulations produced significant results, 
but of these three, only the rating of student quality accounted 
for more fian marginal variance. The lack of significance in 
the self-r ting by attribution crosstabulation may be due, in 
part, to the fart that there were no "worst" ratings in the 
self-evaluations. Observed and expected values were thus the 
same. A review of the contingency table for this variable also 
shows very small residual figures (the largest is -1.8). Figure 
6 presents the graph for attributions by student quality. 



insert Figure 6 about here 
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ATTRIBUTIONS BY CLASS QUALITY 

Faculty Responses 
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Three facts are critical to interpreting this graph: 1) the 
population percentages of internal and external attributions are 
very different from those in the student data; 2) internal 
attributions here, are those in which the instructor claims to 
have been "...the most important factor deteri.iining student 
performance..."; and 3) the factors which influence attributions 
about others are different than those which affect attributions 
about self. These facts suggest that the patterns in these 
graphs can not be compared to those from the student data. 

The most salient feature in Figure 6 is the spread of internal 
attributions across the graph. Interestingly, instructors who 
rate their students among the best also feel that they have 
been instrumental in their students' performance. The extent 
of internal ity among these instructors is 2.25 times the ex- 
pected percentage. As student quality decreases, to "better 
than average" and then "average", instructor ratings become more 
external. Interestingly, although there were 37 ratings of 
"worse than average" classes, no instructor made an internal 
attribution at this level. Although it is tempting to interpret 
the reversal of direction of the internal graph at the "worst" 
casj, as meaningful, the fact is that only 5 "worst" ratings were 
given and four of the five attributions were external. Thus we 
can not assume that in these cases, instructors 'blame' either 
themselves, the students, or circumstances when they perceive a 
given group of students to be generally poor. 



DISCUSSION; 



Success, failure, and student performance attributions. 

Student attributions about success or failure result from a 
complex interaction of events and variables specific to the 
academic situation in question (Gigliotti & Buchtel, 1989; Marsh 
et. al., 1984; Weiner, 1974; 1979). Additionally, these 
attributions are affected by the history of the individual in 
similar or related situations (e.g., by one's perceived 
"efficacy": Bandura, 1977; 1982), by any "dissonance" 
(Festinger, 1957) between these perceptions, and by other 
temporal events outside the college cla sroom and/or beyond the 
institution. 
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In this study, student attributions about academic performance 
were investigated with respect to student ratings of instruc- 
tion with the primary emphases on expected outcomes (grades and 
the amount Issrned), on general student assessments of teacher 
and course (summary ratings), and on personal issues (prepared- 
ness and perceived importance of the subject). 

It is clear from the results of the study that external attri- 
butions, poor grades, and/or negative ratings co-occur with far 
greater frequency than would be expected by examining the simple 
frequencies of internal and external attributions in the sample. 
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It is also clear from the results of the study, that the ratings 
of students whose locus of attribution is external are different 
from those whose locus is internal, and further, that the dif- 
ference is directional: "external" mean ratings are lower than 
"internal" ratings for every variable investigated. Why is this 
the case? 

General attribution theory (Weiner, 1974) can account for the 
coincidence of external attributions and poor performance. Other 
studies {e.g., McMahan, 1974) have reported that college students 
make external attributions after failure. In fact, a replica- 
tion of the failure/externality finding was a necessary condi- 
tion for the rest of the study. Of more interest, was the co- 
incidence of external attributions with evaluation ratings and 
the significant differences between the internal and external 
groups. These require further explanation. The question of 
consequence from the evaluator's point of view is whether at- 
tributional locus is a factor which biases student ratings of 
instruction. For the personality researcher, the question is 
whether student perceptions of faculty performance or other 
factors influence the expected pattern of attributions. These 
two questions raise an important issue: whether attributions are 
a cause or an effect. The authors believe that the results 
support the position that attributions result from performance 
and perceptions. Attributions are effects, not causes. 

Recent evaluation literature {Cohen, 1981; 1987) has shown 
student ratings to positively correlate with successful outcomes 
(i.e., ratings from multiple sections of courses correlated with 
results from an exam common to all sections). If ratings are a 
measure of satisfaction with instructors and courses, then a 
relationship between success and satisfaction should be ex- 
pected. Factor analyses of the TCEP questionnaire (see 
Attachment A) have consistently produced a "course" factor which 
includes the overall course rating and items concerning 
textbooks, assignments, integration of activities, amount 
learned, etc. These items intercorrelate regularly (Cronbach 
'alpha' reliabilities of the "course" scale have always been .80 
or better). Ratings of the instructor, while they always 
correlate strongly with ratings of course and amount learned 
usually fall into the "instructor" factor which is specific to 
instructor teaching skills (reliabilities always .90 or 
better). Thus, at least with TCEP, it is to be expected that 
positive ratings of a course will most often be accompanied by 
positive ratings of the instructor. If success, satisfaction, 
good ratings, and internal attributions are related, and if the 
relationship of failure and external ratings has been es- 
tablished, then will there also be a relationship between 
failure and poor ratings? 

"Grade inflation" 

The usual distribution of grades is skewed toward the positive 
end of the scale. The distribution of expected grades in this 
study shows over 79% to be "A" or "B". The correlation of 
O expected grades to instructor ratings in this sample is .2189 
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(unit of analysis was the individual; results significant at 
<. 00004). The correlation of ACTUAL grades to ratings is .1882 
(unit of analysis was the class with mean grade and mean in- 
structor rating correlated; sample = about 6,000 courses in- 
cluding data used in the present study and results from a short 
questionnaire using a 10-item subset of questions from TCEP; 
results significant at <. 00004). These figures are very similar 
to those reported by Cohen (1981). The effect sizes of these 
correlations are small (under 5% of the variance accounted for) 
but the correlations are significant due, in part, to the issue 
of limited range discussed earlier in this report (see p. 12). 
In this sample, 95% of the expected grades and instructor ratings 
were in the top three categories. Given the relationship between 
satisfaction and success, the significant correlation is thus, no 
surprise. However, a review of the crosstabulation of expected 
grades and instructor ratings shows another interesting result. 
The distribution of responses across the "worst" instructor 
rating shows that "A" students accounted for 20.3% of the ra- 
tings; "B" students accounted for 39.4%; "C" students ac- 
counted for 28%; "D" students, 8.5%; and "F" students 2.4%. 
Failing students did not penalize the instructor for their 
grades! It was the better students who were the most critical. 
Therefore, the reason that the correlations were not larger was 
that they existed in only part of the sample. These results 
also help to clarify the poor prediction for the external/stable 
group (see Table 8 Part III: p. 22). Prediction of membership 
in this group was poor because failing students (the ones most 
often external in their locus) did not "take it out" in the 
instructor. Students expecting all grades rated the instructor 
poorly, especially those students who would be least likely to 
do so if grades and externality were the only variables of 
consequence. 

The conclusion which must be reached is the same as that of 
Howard and Maxwell fl980; 1982): success and satisfaction are 
related but grade inflation is not a viable explanation of this 
relationship. Further, the conclusion supports that of Harsh 
et. al. (1984) and Gigliotti and Buchtel (1989) that though self 
serving bias does exist, it is not a major problem to the 
validity of ratings or the processes of collecting, analyzing, 
or interpreting them. The significant difference between 
internal and external mean scores on evaluative ratings must 
thus be interpreted to mean that while those who perform 
poorly often make external attributions they don't necessarily 
rate their instructors poorly. 

« 

The influence of instructor and course variables on attributions. 

A major portion of this report is devoted to the ways in which 
attribution patterns fluctuated when other variables were 
brought into the analyses (research question # 3). Fig. 4 (p. 
16) shows the extent to which student perceptions of the qua- 
lity of instruction affected their attributions. The same pat- 
terns appeared with "overall course", "amount learned", "course 
organization", "value of content", and "prior preparation". The 
conclusion to be drawn from these results is that student at- 
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tributions take more than expected grades into account. Weiner 
(1974) has used the term "naive psychology" to refer to everyday 
cai:se-and-effect analyses. The present stud^ suggests that in 
academic settings, at least, students are sophisticated in their 
attributions, reflecting the entire situation rather than simply 
their own performance. 



Predicting internal ity and externality 

Attempts to p -sdict membership in internal/external and/or 
internal/external by stable/unstable groups met with modest 
success. Discriminant analyses produced significant results 
and accounted for 10% to 14% of the variance. Results from 
other analyses would seem to suggest that prediction would have 
been more robust. The results may indicate; 1) that predicting 
attributional membership is difficult and imorecise; or 2) that 
modest predictive power was the res(alt of having too few groups, 
limited range of the data, or the intercorrelations between the 
variables used for prediction. 

Long term attributions 

MSLQ results and the lack of attributive differences across all 
levels of grade-point-average suggest that long term causal 
perceptions can remain stable even when specific instances 
provide contradictory evidence. This disputes Owen & Froman's 
(1977) conclusion based on the 'discrepancy' between GPA and 
expected grade. In other words, one's perception of 
one's "efficacy" or one's "expectancy for success" is not 
seriously affected by one instance of a dissonant result. 
However, it seems clear that a series of negative results will 
gradually change these perceptions and in severe cases, (e.g., 
after several failures which can only be internally attributed 
to ability) can lead to a feeling that the individual is 
powerless to succeed no matter how much effort is expended, 
(i.e. "learned helplessness"; Dweck & Goetz, 1978; Garber & 
Seligman, 1980). Attributions are thus 'state' variables with 
respect to specific instructional events but more 'trait' 
variables with respect to predisposition or long-term views of " 
self or situations. 

Faculty attributions 

Several factors make interpretation of the faculty results 
difficult. The first is that, in this sample, the 
co-occurrences (cell sizes) for certain conditions (e.g., 
internal attributions when classes were rated "worse than 
average") were too small to allow meaningful analysis 
and interpretation. The second is that faculty rated their 
classes as groups rather than making attributions about 
individual performance. The third is that attributions about 
others are considerably different than those about self. Weiner 
(1974) interprets research on the differing perceptions of 
teachers, observers, and students (p. 194) stating: 
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fore greatly differ fron; the expectations of others in 
situations where effort is believed to be an important 
causal determinant of outcome. ...Teachers. ..believe that 
they are more responsible for the performance of pupils 
exhibiting improved performance over time than for the 
outcomes of pupils whose performance progressively 
deteriorates. ...Apparently, ego-enhancing and ego-de- 
fensive attributions are likely to be made when one is 
directly involved with the success or failure of others. 

Weiner (p. 198) also presents experimental results showing how 
teachers evaluated student performance in four manipulated 
situations. Students were described as having ability or having 
no ability, and as being motivated or unmotivated. Analysis of 
the rewards and punishments received by these students showed 
that the group receiving the greatest rewards and least punish- 
ment was the 'motivated' group with' 'no ability'. The mo^f 
harshly treated group was the 'unmotivated' group with 
'ability'. He also suggests that those with hiyh achievement 
motivation may be most severe in their attributions about those 
they percieve to be less motivated, 

A note should be made that if the "rules" change when faculty 
make attributions about students' performance, {because those 
attributions are about someone else) then student attributions 
about faculty performance {as reflected in ratings) may follow 
the new "rules" more closely. Do high achieving students expect 
more from instructors? Are they less tolerant of failure to 
provide them with the success, achievement, or learning they 
want? Is this the reason that 59,7% of the "worst instructor" 
ratings in the present data came from 'A' and 'B' students? 

For faculty attributions in this study (see Fig. 5, p. 24), the 
motion of the internal curve as student quality ratings decrease 
is exactly as suggested above. It is a fair assumption that 
college faculty are generally high in achievement motivation. 
Weiner's note about the attributions of high achievers seems 
appropriate to college professors, especially in the context of 
this study. But are all instructors equally motivated? A 
crosstabulation of attributions by faculty rank was performed to 
investigate for attributional changes. While the results were 
significant (see Table 10, p. 24), the only ranks to show much 
variance from expected values were assistant professors (who 
were 1.75 times as internal as expected) and teaching assistants 
(who were .58 times as internal as expected). These figures 
should be interpreted with caution because the overall distribu- 
tion of internal/external attributions was 12% and 88% respec- 
tively. Thus, with only 46 TAs responding, the expected number 
of internal attributions would be rounded to 5. There were 
three, thus the (unrounded) value of .58. Another way to view 
this caution is to note that the value for external attributions 
for TAs is only 1.07 times what was expected. 

When instructor attributions were crosstabulated with instructor 
self-ratings and then with course ratings, very similar patterns 
<j emerged. Instructors were slightly less internal than expected 
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with "best" ratings; slightly more internal with "better" ra- 
tings; somewhat less internal with "average" ratings; and essen- 
tially as expected with "worse" ratings. There were no "worst" 
self -ratings and only 2 "worst" course ratings. As Table 10 
shows, the significance of attributions by course ratings vas at 
a more lenient probability (.02) and the attributions by 
self-ratings were significant only at an extremely lenient level 
(probability =.22), 

On the whole, instructor attributions seem to follow patterns 
suggested by previous research. The present results indicate 
that instructors place value on the effort which they perceive 
has been expended by students and take student ability into 
consideration. Whether their perceptions of the value or 
difficulty of their courses, or their views on the nature of 
course content (i.e., if a topic is particularly interesting or 
vital) affect these attributions can not be estimated at this 
time. The implications of the results have most to do with 
grading and their estimates of students' ability and effort. 
Just as Crittendon & Norr (1973) viewed student ratings as 
"person perception', it may be that grading students is a very 
similar process. If this is true, then should instructors be 
more aware of and more frequently use those strategies which can 
increase student motivation? Doing this could affect student 
performance in the short run by raising levels of achievement. 
In the long-term sense, a series of more successful course 
experiences could raise students' perceptions of efficacy and 
expectancy for s^uccess thus making the student more internally 
oriented and more willing to accept responsibility for his or 
her own performance. 



FUTURE RESEARCH 



ERIC 



Although we know a lot about how students evaluate instruction 
and the contextual factors which may affect ratings, our 
knowledge is less clear about the relationships of personality 
and cognitive variables to ratings (HcKeachie et. al. 1986; 
Mikulincer & Nizan, 1988; Yarbrough, 1989). Understanding these 
variables is important to understanding evaluation but it is 
equally important to the development of effective methods of 
instruction (Keller, 1983) and thus, to student success and 
retention. It seems clear that externally oriented students, 
in many cases, are also students who are at risk. Can the 
performance of these students be improved or, failing that, can 
we ascertain which characteristics are associated with student 
success in specific areas? Certainly, ability and effort are 
paramount, but ability needs to be better defined and understood 
and effort can be enhanced. As Forsyth & McMillan (1981) note: 

By emphasizing the importance of internal, controllable 
factors as causes, teachers may promote pupils' educa- 
tional experiences that are both more satisfying and 
O . more effective. (p. 19) 
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Future research and development should be aimed at these 
targets. The techniques for such research have been described 
(Cross & Angelo, 1988; Gray, 1989; Stark & Mets, 1988) and are 
suitable to single classroom investigations by teachers or to 
larger efforts which could be coordinated through such agencies 
as offices of evaluation and development, institutional 
research, or campus research centers. Established systems for 
campus-wide data collection {such as systems for teacher-course 
evaluation) can provide practical mechanisms for the logistical, 
data management, and analysis needs of such research. Further 
work must also be done on understanding faculty perspectives on 
issues that relate to student ratings as well as faculty and 
administrative understanding and use of student ratings 
(Franklin & Theall, 1989). 



SUMMARY; 



This study attempted to assess the frequencies of internal and 
external student attributions about their academic performance, 
and to determine if these attributions were affected by 
variables which are usually part of the instructional context. 
The analyses used responses from a teacher-course evaluation 
instrument, a measure of motivation, orientation and study 
habits/skills, and a faculty, self-evaluation instrument. 

Three kinds of analysis were conducted in order to address the 
research questions about attributions, grades, and ratings. The 
questions were distinct, but interrelated, and the conclusions 
made about the separat-:^ parts of the study were supported by the 
fact that the three analysis methods produced complementary 
results. Crosstabulations, analyses of variance, and discrim- 
inant analyses supported the convergent and discriminant vali- 
dity of ratings as well as the existence of attributional 
classes and the effects of instructional situational variables 
on them. 

Attributions of performan»:e were shown to be strongly related to 
expected grades in ways predicted by previous research and 
further, to fluctuate with respect to many other instructional 
variables in similar and pr&dictable ways. Additionally, the 
attributions of internally and externally oriented students were 
shown to be significantly different. Attributional (self- 
serving) bias was established but found not to invalidate 
ratings. Some possible r'jasoiis ror attributional patterns were 
presented. Attributions were shown to be effects, not causes. 
Attributions were stable when made vis-a-vis long term 
constructs but varied when made about specific performance in 
courses. And finally, faculty attributions followed patterns 
suggested by previous literature. 
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OfSct of bistnicUonal OtvtlopmtiiUnd'EMiuiiiQfL 4170oda» • ? I 




Which of the fOlbwing options best desaaeilfow^^ statement is 
true regarding YOUlfuSructor in THIS cfesy ' '^TvW^ statement is 



. A s almost always ^^'^ 
B s more than half ofthe time - • 
C « about half of the tune 
D» less than half of the time --• 
E ss ahnost never OR " 

F » this ITEM DOES NOT APPLY to this coarse 
Ihelnstnicton 

t commuisicates the purposes of class sessTons andmstrudional activities. 
2! speaks dearly and audihly whan pn^fffrg rnfnrmnrion. 

3. presents mformation at a rate I can Wawj. 

4. iadicatcs which information is essential and\vhidi U mmor. 

5. uses esamples and Ulustralions which help clarify the topic being discussed. 

6. shows unpqrtant relalioriships among the topics bemg Ueated in this 
'course. . 

7. inspires exdtemenl or interest in the subject matterof this course. 

8. relates course material to relevant real life situations ^en possible. 

9. asks questions '/hich challenge me to thmk! 

10. provides opportunities for me to bring up or discuss issues related to the 
course. 

11. develops an atmosphere of respect and trust iathe classroom. 

12. manages classroom discussions so that they are a useful part of my 
leammg experience. . ^ 

13. presents activities and materials appropriate for my level of experience 
and ability. 

14. clears up points of confusion for me. 

15. provides assistance on an individual basis outside of class when I need it. 

16. gives me regular feedback about how vfdi I am doing in the course, • 

17. states in advance precisely how ray performance is to be evaluated. 

18. gives tests (exams) that are fair and accurate measures of course skills 
concepts, and information as taught. ' 

19. returns exams and assignments quickly enough to benefit me. 

20. suggests specific ways I can improve my performance m this course fwhen 
needed). . 

21. makes effective use of class time. 



ERJC 22. is punctual in meeting class and office hour responsibilities. 



Compared (o other college courses you have taken 



73. The workload for this course is: 

A. one of the lightest 

B. lighter than average 
C about average ' 

D. heavier than average 

E. one of the heaviest- 

24. The difficulty level of the course activities and materials is: 

A. extremely easy 

B. easier than average 
C about average 

D. more difficult than *. /erage 

E. extremely difficult. 

25* The teoabook(s) and readings used in this course are: 

A. among the best 

B. better than average 

C. about average 

D. worse than average 

E. among the worst 

F. item not applicable, no textbooks or readings used- 

26. Rate howwel! the syllabus, course outline, or other overviews provided by 
the instructor helped you to understand the goals and requirements of 
inis course. 

A. unusually well 

B. better than usual 

C about as well as usual 

D. worse than usual 

E. not at all or no such information was provided. 

27. Rate the usefulness of the outside assignments (writings, reports, and 
special projects) m helping you to learn. 

A. extremely useful 

B. more useful than average 

C. of average usefulness 

D. less useful than average 

E. almost useless 

F. item not applicable, no outside assignments 

various elements of the course (e.g., class activities, text- 
DooKs/readings, and outside assignments) worked together in heloin^ 
you learn. ° w.^y*,.^ 

A. very well 

B. better than average 

C. about average 

Q D. worse than average 

ERIC very poorly 55 



29. The course goa's or objectives presented by Uie instructor were met 

A. strongly agree., . ' '• 

B. agree more than disagree" "p ".. 

C. agree and disagree, uncertain' ' 

D. disagree more than agree 

E. strongly disagree 

F. no goals or objectives were presented by the instructor 

30. Overall, how much do you feel you have learned in this course? 

A. an exceptional amount 

B. more than usual 

C. about as much as usual 

D. less than, usual , ^ ■ 

E. almost nothing 

31. What is your overall ratmg of this.ihstructor's teachme effectiveness 
compared 'with other college mstruciors you have hao? 

A. oneofthemo^eSiectiye 

B. more effective than average 

C. about average; .- • . 

D. less effective than average 

E. one of the least effective 

3Z What is your overall rating of this course? 

A. one of the best 

B. better than average \ 

C. about average 

D. worse than average 

E. one of the worst 



Using the scale below. Indicate your agreement or disagreement with the 
following statements about yourself. 

A. strongly agree 

B. somewhat agree 

C. mixed feelings, (agree and disagree) 

D. somewhat disagree 

E. strongly disagree 

F. no opinion or do not understand the question. 

33. My educational background prepared me with the skills and information I 
need to achieve success in this course. 

34. !n my own judgment, what I am being asl^ to learn in this course is im- 
portant. 

35. Overall, I tried to do my best to meet the requurements of this course. 
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36. 



jnw single most important factor detcnnihihgtlu&grade rcxp«t to receive 
lauuscoiiise lias been: ... ' ^ 



A. my ability 

B. my effort 

C the instructor's teaching ability 

D, how difficult (or easy) ue course was 

^ the other students in the course 

P. i&ainlyluck- 

37. lamyprogram, this course is: 

A. required -AND in my major area of study 

B. rMuired-BUTNOTinmymajbrareaofstudy 
C elecUve - AND in my major aieaof study 

D. elective -BUT NOT in my maibr area Sstudy 

E. other (e.g., non-credit or audiQ 

38. Mydassis: 

A. freshman 

B. sophomore 

C middler (for five year and/or *co-op' programs) 

D. junior «^ cr / 

E. senior 

F. graduate student 



39. 



gradepoint average is: (first quarter freshmen use high school 



A, 1.00-1.75 

B. 1.76-Z25 
C 2.26-Z75 

D. 176-3.25 

E. 3.26-4.00 

40. I expect to receive a grade closest to: 

A. A 

B. B 
C C 

D. D 

E. F or U (fail or unsatisfactory) 

F. S (satisfactory, pass) 



YOUR WRITTEN COMMENTS ARE ALSO WELCOME IJ 

Use a sheet of blank paper to write any comments you may haye about 
your mstnictor, this course, or the TCEP questionnaire. Thank vou for 
participating, /' = 



SUMMARY OF FACTOR ANALYSES OF THE 
TEACHER-COURSE EVALUATION PROJECT (TCEP) 
STUDENT RATINGS QUESTIONNAIRE 



This Table presents a composite of over 30 analyses from a total 
sample of 2380 courses (n = 48,500 students). Analyses were 
completed at various times as the sample was gathered; used 
both individuals and classes as units of analysis; subdivided 
the data by class level, GPA level, instructor rating level, 
and sample quality (e.g., a special analysis of classes with 
fewer than five responses and/or less than 50% response rate). 

In all these analyses, the results were literally identical. 
The only variations were; 1) the occasional division of the 
combined feedback and testing factor into two, separate 
factors; and 2) the occasional reordering of the factors (e.g., 
the feedback factor sometimes appeared before the factor 
containing general, course ratings and sometimes appeared 
after the course factor). 

The Table presents the factors with their TCEP items by number. 
Please refer to the TCEP questionnaire for the sepcific item 
wording. Factor loadings, reliabilities (Chronbach 'alphas'), 
and %s of variance accounted for by factors are representative 
of the set of analyses rather than any one, specific analysis. 



FACTOR 


ITEMS 


LOADINGS 


RELIABILITY 


% var. 


Instructor 


1-14, 
21, 22, 
31 


.65 -.80 


.93 


50 


Course 


25-28, 
30, 32 


.55 -.80 


.86 


6 


Feedback & 
Testing 


15-20 


.45 -.79 


.75 


5 


Workload & 
Difficulty 


23-24 


.82 -.92 


.75 


2 



NOTES; 1) Some items loaded on more than one factor but the 

differences in loadings made factor assignment simple. 
The overall instructor item (31) loaded strongly on 
both the instructor and the course factors but 
regularly loaded more heavily on the instructor 
factor; 

2) Item # 29 was invalid and is not included above 
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AND 

THE ORIGINAL AND REVISED ATTRIBUTION 
ITEMS AND SCALES 
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. survey of attft:udes», motlvaj 





I. I prefer course work that 1$ challengin^^sa^at E 'cin' learn new thlngr^ 
Z. If I want to do get good grades in my wOursesV. Vtlle^ on what I do/^J 

3. Compared with other students in my classes, t think I have excellent o^^SS 
writing skills. ..r.,- • - - , " 

4. I usually like the subject matter of my courses. . . 

5. I think that what I learn in one course,.!; can use In other courses. -f-^W' 

6. I believe I will receive excellent grades in most of my courses. 



I'm certain I can understand the most difficult raaterial presented in 
the readings from my cc trses. - v^" ^^4fePC/^ 

8. Getting good grades is my main goal in my courses, ^ - u^-'^^^^Cr 

9. When I take a test I think a lot about the items that I can't answer. . '^B^^M:-' 

10. It is ray own fault if I don't do well fn my courses. 

11. It is important for me to learn the material in my ^courses. 

12. Compared with others in this class, i: think I. have excellent study skills 

13. I'm confident i can learn the basic concepts taught in my courses. 
M. If I can, I want to get better grades than most other students. 

15. If I get good grades in my classes, it is because of the instructors' 
teaching skill. 

16. When I take tests, I think of the consequences of failing. 

17. Compared with others in .-ny courses, I think I'm a good student. 

18. I prefer course work tnat arouses my curios'ity, evon if it is difficult. 

19. I am usually quite intersted in the content of J5y courses. 

20. If I don't do well in rny courses, it 'is because' the instructors aren't 
going a good job of teaching. 

21. I have an uneasy, upset feeling when I.^itake an exam. 

22. I'm confident I can do an excellent Job on the assignments and tests 
in my courses. 

23. I expect to do well in .-ny courses. 

24. Hy main goal in this course is to learn a great deal about the subject. 

25. I think the material In my courses i$ useful for me to learn. 

26. When I have the opportunity, I choose course assignments that are 
challenging to me even if they don't guarantee a good grade. 



27. Compared with others in .-ny courses, I- think I have a good background 
in the courya c ontent. ^ 



I, J. . 
^ ■ . » 



28. -If ITgtt poor grades 'In my courses', it it. twciust tht fnstructors: .^^m 
don't like students- Vike me. •■ -hsHv' - -^'-^^ 

29. Understanding the subject mttter of My courses it very, inportant to me. 

30. If I want to do well in my courses, it wfll depend on how hard I try. 
31* I'a certain I can master the skills being taught in ^y courses. 

32. If t do well in my courses, it's due to the easy grading system. 

33. I prefer easy and familiar course aaterlal so t can get good grades. 

34. Compared with* other students In oQf courses,: I! think twill do well. 

35. Uhen I take a test I think about how poorly Cam doing compared with 
other students. - 

36. It is inportant for me to get good grades to iiprove qy career prospects. 

37. I'a confident I can understand the onst conplex laterial presented by 
•y instructors. 

38. If I get poor grades, it Is because f didn't try hard enough. 

39. I think my reading skills are excellent compared to other students. 

40. When I study for a test, I try to reraember ai much detailed information 
as possible. \ 

41. I often find that I don't spend very Bsuch tine on my courses because of 
other activities. 

42. When studying, I try to determine which concepts i: don't understand well. 

43. When confronted with d1fflc,;t aaterlal or problens, I try to think up 
possible solutions and then chock then out. 

44. When I become confused about something I'» reading, I ge back and try to 
figure it out. 

When I take notes, I try to write down as nuch as possible of what the 
instructor jays. 

46. I nake good use of my study time. 

47. If course readlnos are difficult to understand, I change the way I 
read the naterial. 

48. I have difficulty identifying the inportant points in ny readings. 

49. When studying for a test, I read and reread ny class notes'* and 'the 
course readings. 

50. When a theory, Interpretation, or conclusion is presented, I try to 
decide if there Is good supporting evidence. 

51. I work hard to get a good grade even when I don't like a course. 

52. I memorize definltlonj when studying for my courses. 

-- 6J 
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53. J try to do. course work on my owfr,- without help. V. 

55. I find U hird to stick to a study schedule. 

rcis^L"?^ iri^arrel; feSdl W^^l^^^^^^^^^ ^^^^-nt 
?l^%iiler " thoroughly. I. often skim it to see how it 

belf sJJIdJlJg^"'''*^"' '° ^ understand the material I have 

^ I *?*Pt my st^cJy1ng and learning style to- the Inst ctor's 

requirements and teaching style. "^^ '^"'^ ^ 

60. When I study for a tast. I copy my notes over. 

61. I try to identify students in class whom I can ask for help if necessary. 

62. If course work is difficult, I give up or only study the eisy pa..s. 
JSoCghts?"^' ' ^ •"^^^'•^^^ to help me organize my 

L™J^7l'"fIf diagrams, or tables to help me organize or 

summarize the course material. wiyom^c or 

65. I have trouble taking notes unless lectures are well organized. 

66. When reading. I try to relate the material to what I already know. 

67. I have a regular place set aside for studying. 

nZll ?l*^y "Sunes!^' ^'"^ °' ''''''' ^ - 

69. I write brief summaries of the main Jdeas in my lecture notes. 

70. When reading for a course. I make up questions tc help focus my reading. 

l^r^ Jn"3^[h?2Snc^^S[!%rtt^ett?^ ^^"""^^^^ the 
72. I make sure that I keep up with weekly readings and assignments. 
{hJrtrrnSrreqSfrlJ'^'^'" snd-of-chapter questions even If 

74. I attend class regularly. 

75. I >nemorize key worcs to remind me of Important concepts on a test. 

76. When I study, I practice saying the material to myself over and over. 

77. ! often find myself qjestionlng things I hear or read in course tn 
decide If I find th C'n convincing. 
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79. r«k tht instructor" to- clarify conceptr tdbn't undtrstand well. 

^n"d^%^7^;to^„:^^o^r?o^^l??^^55 fipXKsjiJ!^" " 

82. I rirtly find time to review my notes or rtidings tsfore an exam. 

83. - When I study, I try to think through a topic and decide what I am 

supposed to learn from it. rather than Just rSdinS It oJe?. 

SIlS"ide«!^* * r usually include some of my 

85. I usually study in a place where I can concentrate on my course work. 
S^ifcSaTJJS^o^c^Jf::"'^' ' bacic oyer ^ notes and fill 

ihen'ive? p^sJble''" ''''^ ^" ''''' 

Mjr1JpJc"{s^'f'i;°7lJl^' "^'^^'^ I" »y 

f?^"ndMsSltr ^ informatioVto a 

91. When I study a topi.. I try to make everything fit. together. 

92. If I get confused taking notes in lecture, I make sure I sort it out 
as soon as possible after class. ^ 

it'JlranlbS!!?! ^ ''''' """^^"^ class but donH know what 
94. When I study, I try to remember as much detailed information as possible. 
'on(rJ2l?'"^?Se^JJ?.c"?J?^rfd11;^ '''''' topics-rathpr than to 
'JhS'le%\"u1e1*a^d"read"ig"'' " '''''''''' 

^SrSthlflhlngs! ""^^ ^"^"'^ -"^ ^'""'^ ^ ^'^^"'^'"9 

SSa^of^abint?"^^^"^ ^'^'^ course.work would be admitting my 

99. I believe thst people would think less of ne if I got help in order 

to succeed in my courses. *^ 

100. I would think less of myself If I couldn't do my course work 
without help. 
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HSLQ ORIGINAL AHRIBUTIONAL ITEMS AND RESPONSE SCALE 

AND 

PARAPHRASED ITEMS AND RESPONSE SCALE 



« 



MSLQ 2.. If I want to get a good grade in this course, it depends 
what I do. 

Study 2.. If i want to get good grades in my courses, it depends 
on what I do.. 

MSLQ 10.. It is my own fault if I don't do well in this course. 
Study 10.. It is my own fault if I don't do well in my courses. 

MSLQ 15.. If I get a good grade in this class, it is because 

of the instructor's teaching style. 
Study 15.. If I get good grades in my classes, it is because of 

the instructors' teaching skill. 

MSLQ 20.. If I don't do well in this course, it is because of 
the instructor. 

Study 20.. If I don't do well in my ~ourses, it is because the 

instructors aren't doing a good job of teaching. 

MSLQ 28.. If I get a poor grade in this class, it is because the 

instructor doesn't like students like me. 
Study 28.. If I get poor grades in my courses, it is because the 

instructors don't like students like me. 

MSLQ 30.. If I want to do well in this course, it will depend on 
how hard I try. 

Study 30.. If I want to do well in my courses, it will depend on 
how hard I try. 

MSLQ 33.. If I do well in this course, it's due to the easv 
gradi'ig system. 

Study 32.. If I do well in my courses, it's due to the easy 
grading system. 

MSLQ 39.. If I get a poor grade in this course, it is because 

I didn't try hard enough. 
Study 38.. If I get poor grades, it is because I didn't try hard 

enough. 

MSLQ scale: 

Very true of me X X X X X Not at all true of me 
Study scale: 

Very much like me. . .Somewhat like me... More like, than unlike me 
More unlike, than like me. . .Somewhat unlike me... Very much unlike m 
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Instructor's Questionnaire 



Office of Instructional Development and Evaluation 









mmm 









Use the scale below to indicate liow often st&temeiits 1 througli 22 are true 
about YOU as an Instructor for THIS class. 

A = almost always 

B = more lhan half of the time 

C = about half of the tune 

D = less than half of the tune 

E = almost never or 

F = this item DOES NOT apply to this course 
As the instructor, I 

1. communicate iht purposes of class sessions and learning activities. 

2. speak clearly and audibly when presentmg mformation. 

3. present information at a rate students can follow. 

4. indicate which information is essential and which is minor. 

5. use examples and illustrations which help clarify the topic being discussed. 

6. show important relationships among the topics bemg treated in this course. 

7. inspire excite.i:<jnt or interest in the content of this course. 

8. relate course material to relevant, real life situations when possible. 

9. ask questions which challenge students to think. 

10. provide opportunities for students to bring up or discuss issues related 
to the course. 

11. develop an atmosphere of respect and trust in the classroom. 

12. manage classroom discussions so that they are a useful part of students' 
learning experience. 

13. present activities and materials appropriate for students' levels of 
experience and ability. 

14. clear up points of confusion for students. 

15. provide assistance on an individual basis outside of class if students need it. 

16. give students regular feedback about how well they are doing in the course. 

17. state in advance precisely how student performance is to be evaluated. 

18. give tests (exams) that are fair and accurate measures of course skills, 
concepts, and information as taught. 

19. return exams and assignmer its quickly enough to benefit students. 

20. suggest specific ways students can improve their performance in this 
course (when needed). 

21. make effective use of class time. 66 



36. The primary (most important) mode oiTihsthic^on for this course is: '^^vMf^y 

A. lecture (by instructor);and exsu^^ and/or papers ' ^"#f : 

B. group discussion with team or 'dbllaborative projects T\ 
C laboratory, performance, or other %ands on'' in-class activiities • 

D. clinical, field work, or practicum ofF-campus activitie 

E. independent student research wth individual supervision 
R presentations of invited lecturers, videotapes, films, etc 

with discussion, papers, and/or exams. 

37. The main method used for evaluation ofs^.udent performance in this class •: ^ 
is: 

A. tests and/or exams only r\ 

B. papers and projects only "^^^ 
C papers and/or projects and tests and/or exams 

D. performances, presentations, or demonstrations - 

E. non-print projects (e.g. constructions or febrications for 
engineering; paintings, photographs, drawmgsfor fine arts) 

F. assessment of quality of participation in class, groups, or 
team-work. 

38. Compared to other groups of students you have taught, how would you rate 
this ^oup? 

A. among the best 

B. better than usual 

C. about the same as usual 

D. worse than usual - ^ 

E. among the worst 

Use the scale below to indicate the emphasis you placed in this class on the 
objectives described below. 

A. = very heavy emphasis 

B. = moderate emphasis 

C. = some emphasis 

D. = slight emphasis 

E. = no emphasis 

Students: 

39. gainmg factual knowledge (terminology, classifications, methods, trends). 

40. learning fundamental principuls, concepts, ortheories. 

41. unproving logical thinking, problem-solving, and decision-making. 

42 developing specific psychomotor (kinesthetic, manipulative, or manual) 
skills. 

43. developing-skills in organizing ideas and presenting them in written form. 

44. opportunities to be creative (imaginative, inventive, criginal). 

45. developing a favorable altitude toward the subject matter. 

46. developing skills for leadership, teamwork, and group work. 

V. 3.0 
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29. nie course goals or objectivc^as you ps;esented them were met. 

A. strongly agree • * 

B. agree more than disagree 

C. agree and disagree ahcertam 

D. disagree more than agree ■ 

E. strongly disagree ::• ■ " * " * 

30. Overall, how much do you fee! students leaiaed ia this coarse? 

A. an exceptional amount 

B. more than usual ' 

C. about as mu&h as usual 

D. less than usual 

E. almost nothing 



31. 



What is your overall rating of your teaching effectiveness fin this 
course/section)comparelwth other college ii^S 



A. oneofthemosteflfective 

B. more effective than most 

C. about average 

D. less effective than most 

E. one ofthe least effective 

32. What is your overall rating of this course? 

A. one of the best 

B. better than average 

C. about average 

D. worse than average • ' 

E. one of the worst 
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Please answer the following items: 

33. Your rank: (leave blank if not applicable) 

A. full professor (including emeritus) 

B. associate professor 

C. assistant professor 

D. instructor 

E. lecturer (including adjunct, senior, and part-time) 

F. teaching assistant 

34. Your years of experience teaching 

A. less than one 

B. one to two 

C. more than two but less than five 

D. five or more but less than eiglit 

E. eight or more but less than twelve 

35. Years of experience, con't: 

O ^' »"ore than twelve but less than twenty 

68 




23. The workload for this course is " • " " ' • ' - 'tz^^^ 

A one of the lightest • ' 

B. lighter than average 
C about average 
D. heavier than average 

one of the heaviest. 

24. The difficdty level of the course activities and materials is 

A* . extremely easy . - ' 

easier than average 

C. about average 

D. more difficult than average 

E. extremely difficult ' . ' 

25. The textbook(s) and readings used in this course are ' 

A. among the best 
better than average 

C. about average 

D. worse than average . 

E. among the worst. 

26. Rate how well the syllabus, course outline, or otfier overviews you 
provided may have helped students to miderstand'the goals and require- 
ments of this course. 

A. unusually well 

B. better than usual 
C about as well as usual 

D. worse than usual 

E. not at all or no such information was provided. 

27. Rate the usefulness of the outside assignments (vmting, reports, and spe- 
cial projects) in helping students to leam 

A. extremely useful 

B. more useful than average 

C. of average usefulness 

D. less useful than average 

E. almost useless 

R outside assignments were nc" a significant part of instruction 

28. Rate how v/ell the various elements of the course (e.g., claf s activities, text- 
books/readmgs, and outside assignments) worked together in helping stu- 
dents leam. ^ 

A. very well 

B. better than average 

C. about average 

D. worse thaa average 

E. very poorly 
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